Method and apparatus for training machine learning model for determining operation of medical tool control device

ABSTRACT

A method for training, by a processor, a machine learning model for determining an operation of a medical tool control device may comprise the steps of: obtaining an operation command to move a medical tool of the medical tool control device on the basis of the machine learning model from guide data generated using a blood vessel image; generating evaluation data of a position to which the distal end of the medical tool has been moved according to the operation command in a blood vessel image; and updating a parameter of the machine learning model by using the evaluation data, so as to train the machine learning model.

TECHNICAL FIELD

The following description relates to a method and an apparatus fortraining a machine learning model for determining an operation of amedical tool control device.

BACKGROUND ART

When treating cardiovascular, cerebrovascular, and peripheral bloodvessels, an interventional procedure that inserts a stent and the likeusing a guide wire and a catheter is widespread. The guide wire is atool for setting a path for transporting a stent and the like into theblood vessel through the catheter, and in order to transport the guidewire to the end of the blood vessel with diseases, visual informationbased on medical images such as angiography, tactile information basedon fine feeling of the hand, and the like are utilized.

Recently, remote robots and the like have been developed to reduce thephysical burden of an operator, such as radiation exposure, and toprecisely control surgical tools. Although surgical robots have beencommercialized through the FDA, learning to be adapted to new tools isrequired to perform simple surgical operations. Even if thecorresponding operation, such as moving the guide wire backwards orrotating the guide wire at a predetermined angle, is not directlyperformed, functions that the robot takes over are being added, but theproportion of the operations in the surgical procedure is small.

Prior Art Document

-   (Patent Document 1) KR Patent Application Publication No.    2017-0049172 (published on May 10, 2017)

DISCLOSURE OF THE INVENTION Technical Solutions

According to an aspect, there is provided a method for training, by aprocessor, a machine learning model for determining an operation of amedical tool control device may include the steps of: obtaining anoperation command to move a medical tool of the medical tool controldevice on the basis of the machine learning model from guide datagenerated using a blood vessel image; generating evaluation data of aposition to which the distal end of the medical tool has been movedaccording to the operation command in a blood vessel image; and updatinga parameter of the machine learning model by using the evaluation data,so as to train the machine learning model.

According to an aspect, the generating of the evaluation data mayinclude applying a compensation value calculated according to acomparison result between the position to which the distal end of themedical tool has been moved in the blood vessel image and the guide datato the evaluation data.

The guide data may include information about a position of the distalend of the medical tool together with at least one of a middle targetpoint, an access restriction point, and a destination point in a patchimage corresponding to at least a partial area of the blood vesselimage, and the generating of the evaluation data may include applying afirst group compensation value to the evaluation data in response to acase where the distal end of the medical tool reaches one of thedestination point and the middle target point in the blood vessel image.

The applying of the first group compensation value may include applyinga first compensation value to the evaluation data in response to thecase of reaching the destination point, and applying a secondcompensation value smaller than the first compensation value to theevaluation data in response to the case of reaching the middle targetpoint.

The generating of the evaluation data may include applying a secondgroup compensation value to the evaluation data in response to a casewhere the distal end of the medical tool reaches an access restrictionpoint and a case where the distal end of the medical tool has been movedinto a section between the areas.

The applying of the second group compensation value may include applyinga third compensation value to the evaluation data in response to thecase of reaching the access restriction point, and applying a fourthcompensation value having a smaller absolute value than the firstcompensation value in response to the case of moving into the sectionbetween the areas.

The training of the machine learning model according to an exampleembodiment may include calculating an estimated evaluation valueassociated with an operation command output from a guide imagecorresponding to a current time frame by using the machine learningmodel, calculating a measured evaluation value (target evaluation value)from a guide image corresponding to a next time frame after the medicaltool has been moved according to the operation command output in thecurrent time frame, and determining a parameter to update the machinelearning model by using the estimated evaluation value and the targetevaluation value.

The obtaining of the operation command of the medical tool controldevice according to an aspect may include calculating an expectationvalue to reach a destination point for each of candidate operationcommands that may be performed by the medical tool control device byusing at least a part of the machine learning model, and outputting anoperation command having the highest expectation value among thecandidate operation commands by using the remaining machine learningmodel.

Further, the method for training the machine learning model may includecalculating a blood vessel structure from the blood vessel image,setting a middle target point and an access restriction point based onthe blood vessel structure and the destination point, obtaining anentire guide image in which the destination point, the middle targetpoint, and the access restriction point are set, extracting partialguide information from the entire guide image based on the position ofthe distal end of the medical tool, and determining the operationcommand based on the machine learning model from the partial guideinformation.

Advantageous Effects

According to the method for training the machine learning model fordetermining the operation of the medical tool control device accordingto an example embodiment, it is possible to improve the speed of themedical tool control device by controlling the medical tool controldevice using an artificial neural network and learning the medical toolcontrol device using the result.

Further, it is possible to numerically evaluate a series of operationcommands of the medical tool control device by generating a destinationpoint, a middle target point, and an access restriction point.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an operation of a medical tool controldevice according to an example embodiment.

FIG. 2 is a flowchart illustrating a method for training a machinelearning model according to an example embodiment.

FIG. 3 is a diagram illustrating generating guide data from a bloodvessel structure image according to an example embodiment.

FIG. 4 is a diagram illustrating generating blood vessel structure datafrom a blood vessel image according to an example embodiment.

FIG. 5 is a block diagram illustrating determining an operation commandof the medical tool control device on the basis of the machine learningmodel according to an example embodiment.

FIG. 6 is a block diagram illustrating training the machine learningmodel according to an example embodiment.

FIG. 7 is a diagram illustrating a guide image corresponding to acurrent time frame and a guide image corresponding to the next timeframe according to an example embodiment.

FIG. 8 is a block diagram illustrating a system for training a machinelearning model for determining an operation command of a medical toolcontrol device according to an example embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

Specific structural or functional descriptions of example embodimentswill be disclosed for purposes of only examples, and may be changed andimplemented in various forms. Accordingly, the example embodiments arenot limited to a specific disclosure form, and the scope of the presentspecification includes changes, equivalents, or substitutes included inthe technical spirit.

Terms such as first or second may be used to describe variouscomponents, but these terms should be interpreted only for the purposeof distinguishing one component from other components. For example, afirst component may be referred to as a second component, and similarly,the second component may be referred to as the first component.

It should be understood that, when it is described that a component is“connected” to the other component, the component may be directlyconnected to or access the other component or a third component may bepresent therebetween.

Singular expressions used herein include plurals expressions unless theyhave definitely opposite meanings in the context. In the presentspecification, it should be understood that the term “including” or“having” indicates that a feature, a number, a step, an operation, acomponent, a part, or the combination thereof described in thespecification is present, but does not exclude a possibility of presenceor addition of one or more other features, numbers, steps, operations,components, parts, or combinations thereof, in advance.

If it is not contrarily defined, all terms used herein includingtechnological or scientific terms have the same meanings as thosegenerally understood by a person with ordinary skill in the art. Termswhich are defined in a generally used dictionary should be interpretedto have the same meaning as the meaning in the context of the relatedart, and are not interpreted as an ideal meaning or excessively formalmeanings unless clearly defined in the present specification.Hereinafter, example embodiments will be described in detail withreference to the accompanying drawings. Like reference numeralsillustrated in the respective drawings designate like members.

FIG. 1 is a diagram illustrating an operation of a medical tool controldevice according to an example embodiment.

A medical tool control device 110 according to an example embodiment maymove a medical tool 120 to a blood vessel destination point according toa driving command by a processor. For example, the medical tool controldevice 110 may move the distal end of the medical tool 120 to the bloodvessel destination point. The medical tool control device 110 may beimplemented as a robot for performing surgery, and for example, a robotfor controlling the medical tool for cardiovascular intervention.

The medical tool 120 is a member inserted into a blood vessel, and mayinclude a medical tool disposed on the distal end of the medical tool120 and a medical wire connecting the medical tool to a driving unit.The medical wire may include, for example, a catheter or a guide wire.The guide wire may refer to a medical wire used for inserting andguiding the aforementioned medical tool to a target site of the bloodvessel. The medical tool may be a surgical tool operated according tothe control of a doctor, for example, an introducer kit.

The medical tool control device 110 may determine the above-describeddriving command using a blood vessel image. For example, the medicaltool control device 110 may output a driving command from the bloodvessel image by performing a calculation according to the machinelearning model. The machine learning model is a model designed andtrained to receive the blood vessel image and output guide data, and maybe implemented as, for example, a neural network model. However, theinput of the machine learning model is not limited to the blood vesselimage, and may also be a blood vessel structure image and guide data.The guide data may represent data in which guide information is mappedto the blood vessel image or the blood vessel structure image. The bloodvessel structure image is an image in which a specific blood vessel isextracted from the blood vessel image, and may be an image obtained bypreprocessing the blood vessel image. The blood vessel structure imagewill be described with reference to FIG. 3 below. The blood vessel imagemay be an image generated using corona angiography (hereinafter, CAG) ormagnetic resonance imaging (hereinafter, MRI). In the blood vesselimage, not only the blood vessel but also the medical tool 120 may bephotographed.

The guide information is information for guiding the movement androtation of the medical tool 120, and may include, for example,information about an area in which the medical tool 120 should depart,an area through which the medical tool 120 should pass, and adestination point in the blood vessel. The information on each area mayinclude image coordinates in the blood vessel structure image of thecorresponding area, but is not limited thereto. According to an exampleembodiment, guide information may be visually mapped to the blood vesselstructure image. For example, a graphic object corresponding to eachtarget area may be visualized in the blood vessel structure image, andthe blood vessel structure image in which the target area is visualizedmay be represented as a guide image.

The processor of the medical tool control device 110 may determine todrive the medical tool 120 based on the result of analyzing the bloodvessel image. The processor may determine a driving command by using atleast one of the blood vessel image, the blood vessel structure image,and the guide data. The driving command may represent a command foroperating the driving unit connected to the medical tool 120 to move androtate the medical tool 120. The driving command may be, for example, aforward command, a backward command, a clockwise rotation command, and acounterclockwise rotation command, but is not limited thereto.

The medical tool control device 110 may analyze the received bloodvessel image to generate guide data, and the medical tool control device110 may determine a driving command from the generated guide data. Forexample, the medical tool control device 110 may select, as an operationcommand, one of a forward command, a backward command, a clockwiserotation command, and a counterclockwise rotation command from the guidedata. The driving unit of the medical tool control device 110 may bedriven according to the selected operation command. For example, thedriving unit may move the medical tool 120 forward in response to theforward command. The driving unit may retract the medical tool 120 inresponse to the backward command. The driving unit may rotate the guidewire clockwise based on the longitudinal axis of the guide wire inresponse to the clockwise rotation command. The driving unit may rotatethe guide wire counterclockwise based on the longitudinal axis of theguide wire in response to the counterclockwise rotation command.

Accordingly, the medical tool control device 110 determines a series ofoperation commands using the guide data generated by analyzing the bloodvessel image, thereby moving the distal end of the medical tool 120 tothe area to be guided by the guide data. The medical tool control device110 may move the distal end of the medical tool 120 to a finaldestination point by repeating the operation determination using theguide data. After the distal end of the medical tool 120, for example,the medical tool reaches the destination point, the medical tool mayperform a surgical operation under the control of the doctor.

Referring to FIG. 1 , after the medical tool 120 is inserted through theblood vessel of the wrist of a patient, the medical tool control device110 may guide and move the medical tool 120 to a target blood vessel.However, the medical tool 120 is not limited to being inserted throughthe blood vessel of the wrist of the patient, and may be insertedthrough the blood vessel of the lower extremity of the patient.

FIG. 2 is a flowchart illustrating a method for training a machinelearning model according to an example embodiment.

In step 210, a processor for training a machine learning model accordingto an example embodiment may obtain an operation command of a medicaltool control device. The processor may obtain an operation command formoving the medical tool of the medical tool control device based on themachine learning model from guide data generated using a blood vesselimage. The guide data may include information about a position of thedistal end of the medical tool in the blood vessel image together withat least one of a middle target point, an access restriction point, anda destination point in a patch image corresponding to at least a partialarea of the blood vessel image. An example embodiment of generatingguide data will be described below in detail with reference to FIG. 3 .

In step 220, the processor according to an example embodiment maygenerate evaluation data of the obtained operation command. Theevaluation data may be evaluation data on the position to which thedistal end of the medical tool has been moved according to the operationcommand in the blood vessel image. The processor may generate theevaluation data by applying a first group compensation value or a secondgroup compensation value according to whether one of the middle targetpoint, the access restriction point, and the destination point isreached in the blood vessel image including the guide data. That is, theprocessor may generate the evaluation data by applying a compensationvalue calculated according to a comparison result between the guide dataand the position to which the distal end of the medical tool has beenmoved in the blood vessel image. The compensation value may be set todifferent values based on a position, a time, and the number of controlcommands.

In step 230, the processor according to an example embodiment may trainthe machine learning model by updating a parameter of the machinelearning model using the evaluation data.

FIG. 3 is a diagram illustrating generating guide data from a bloodvessel structure image according to an example embodiment.

According to an example embodiment, the processor may obtain a bloodvessel structure image from a blood vessel image 310 and generate guidedata based on blood vessel branch area information of the blood vesselstructure image. However, the present invention is not limited thereto,and the processor may receive an entire guide image 320 in which theguide information is generated in the blood vessel structure image. Theguide information may include a destination point 322 to guide and movethe distal end of the medical tool by the medical tool control device, amiddle target point 331 on a path from a start area 321 to thedestination point 322, and an access restriction point 332 which is anarea to be excluded by the guide of the distal end of the medical tool.

The blood vessel structure image may be an image in which a blood vesselidentified from the blood vessel image 310 and a structure and aconnection relationship of the blood vessel are displayed. The processoraccording to an example embodiment may generate a blood vessel structureimage by dividing a blood vessel area and a background area from theblood vessel image 310 by using an edge detecting method.Illustratively, the edge detecting method may detect, as an edge, anarea in which the grayscale levels of an arbitrary pixel and neighboringpixels rapidly change, but is not limited thereto, and may also beanother method of detecting an edge between the blood vessel area andthe background area.

The processor according to an example embodiment may extract a targetblood vessel from the image divided into the blood vessel area and thebackground area, based on a thickness of the blood vessel in the bloodvessel area and a grayscale level in the image. For example, when it isdesired to extract a cardiovascular system as a target blood vessel, theblood vessel image 310 may include not only the cardiovascular systembut also blood vessels other than the cardiovascular system. In the caseof using the CAG, a blood vessel in which a contrast medium is injectedmay have a lower grayscale level than a blood vessel in which thecontrast medium is not injected, and a blood vessel in which the medicaltool is movable may have a greater thickness than a blood vessel inwhich the medical tool is not movable. Accordingly, for example, inorder to extract the cardiovascular system, the processor may determinea blood vessel having a thickness greater than a threshold thickness anda grayscale level lower than a threshold grayscale level as thecardiovascular system in the blood vessel area. However, the presentinvention is not limited thereto, and the processor may distinguish ablood vessel area to be extracted using a trained machine learningmodel.

According to an example embodiment, the processor may extract partialguide information 330 based on the position of the distal end of themedical tool from the entire guide image 320 in which the guideinformation is generated in the blood vessel structure image. Theprocessor may determine an operation command of the medical tool controldevice based on the guide information included in the partial guideinformation 330. That is, the processor may determine an operationcommand of the medical tool control device based on the partial guideinformation 330 focused on the position 333 of the distal end of themedical tool and the positions of the middle target point 331 and thedestination point 322 to be guided by the distal end of the medical toolaround the distal end of the medical tool. The processor may extract newpartial blood vessel information whenever the medical tool is moved byperforming one operation command by the driving unit of the medical toolcontrol device. The processor may compare partial blood vesselinformation corresponding to the current time frame with new partialblood vessel information corresponding to the next time frame after oneoperation command is performed, and may update a parameter of themachine learning model based on the comparison result.

FIG. 4 is a diagram illustrating generating guide data by generatingblood vessel structure data from a blood vessel image according to anexample embodiment.

The processor according to an example embodiment may generate bloodvessel structure data from the blood vessel image, and generate guidedata based on the blood vessel structure data. The processor may extracta blood vessel area from a blood vessel image 410, and may recognize aposition where a blood vessel branch starts in the blood vessel area asa blood vessel branch area. In a vascular simplified image 420illustrated in FIG. 4 , the recognized blood vessel is marked as a solidline, and a position identified as the blood vessel branch area and thedistal area of the blood vessel are marked as nodes.

The processor may generate blood vessel structure data 430 based on thebranch area identified from the vascular simplified image 420 andconnection information of the branched blood vessel. According to anexample embodiment, the processor may generate node data indicating thebranch area and edge data indicating the branched blood vessel. Sincethe branched blood vessel is connected to two different branch areas,the processor may connect two node data to one edge data, and the nodedata may map edge data corresponding to the number of branched bloodvessels. The connection information of the branch area and the branchedblood vessel may be information indicating a connection relationshipbetween the branch area and the branched blood vessel, and may begenerated using edge data mapped to the node data and node data that isa connection target of the edge data. The processor may data-structurethe blood vessel based on the connection information. For example, theprocessor may generate a tree structure in which the nodes and the edgesare connected by using a branch area closest to a blood vesselintroduction part as a root node. The root node may be a nodecorresponding to the highest branch area, but may be a nodecorresponding to the start area.

Thereafter, the processor may search for a path from the root nodecorresponding to the start area to the node corresponding to thedestination point based on the blood vessel structure data 430, and mayset areas in the blood vessel image corresponding to the nodes on thepath as middle target points. The processor may set all modes except forthe node set as the middle target point among all nodes of the bloodvessel structure data as the access restriction point.

The processor, which determines the operation command of the medicaltool control device based on the machine learning model according to anexample embodiment and trains the machine learning model based on thedetermined operation command, may consist of one or more physicalprocessors. That is, the determining of the operation command of themedical tool control device and the training of the machine learningmodel may be implemented in a device consisting of a single housing, andeach function may be implemented as a function distinguished from eachother in one or more processors. However, the present invention is notlimited thereto, and the determining of the operation command and thetraining of the machine learning model may be implemented by a processorin a separate housing. In addition, a memory for determining anoperation command and a memory for training a machine learning model maybe included in separate housing devices, but may be included in onehousing device. FIG. 5 may illustrate operations of the processor andthe memory with respect to the determining of the operation command, andFIG. 6 may illustrate operations of the processor and the memory withrespect to the training of the machine learning model.

FIG. 5 is a block diagram illustrating determining an operation commandof the medical tool control device on the basis of the machine learningmodel according to an example embodiment.

According to an example embodiment, a processor 521 of a device 520 fordetermining an operation command may receive partial guide information510 of a current time frame. The processor 521 may determine anoperation command for guiding the distal end of the medical tool to themiddle target point or the destination point based on the position ofthe distal end of the medical tool, and the positions of the middletarget point, the destination point, and the access restriction point.The processor 521 may transmit an operation command to a medical toolcontrol device 530. A memory 522 according to an example embodiment maystore the partial guide information 510 of the current time frame and anoperation command in the current time frame determined by the processor521.

The medical tool control device 530 may perform an operation commandprovided from the processor 521 while gripping the medical tool. Forexample, as described above in FIG. 1 , the medical tool control device530 may move or rotate a medical tool (e.g., a guide wire and a medicaltool) by driving a driving unit according to the operation command.

After the medical tool control device 530 executes the operationcommand, the processor 521 and the memory 522 may receive partial guideinformation 550 of the next time frame related to the distal end of themoved medical tool. For example, a blood vessel imaging device 540 mayoutput the partial guide information 550 corresponding to the next timeframe in the next time frame after the medical tool control device 530performs the operation command in the current time frame. The processor521 may determine an operation command of the medical tool controldevice 530 in the next time frame based on the partial guide information550 corresponding to the next time frame. The memory 522 may store thepartial guide information 550 corresponding to the next time frame asone set mapped together with the partial guide information 510corresponding to the current time frame. In the present specification,the blood vessel information about the current time frame and the nexttime frame has been mainly described as the partial guide information510 and 550, but is not limited thereto and may be a blood vessel imageor entire guide information about the current time frame and the nexttime frame.

According to an example embodiment, the processor 521 may determine anoperation command by using the machine learning model. For example, theprocessor 521 may calculate an expectation value to reach a destinationpoint for each of the candidate operation commands that may be performedby the medical tool control device 530 using at least a part of themachine learning model. The processor 521 may output an operationcommand having the highest expectation value among the candidateoperation commands by using the remaining model of the machine learningmodel. The medical tool control device 530 may perform an operationcommand output by the machine learning model.

According to another example embodiment, the processor 521 may calculatedirectly an operation value by using the machine learning model. Forexample, the processor 521 may output an operation command from inputdata (e.g., partial guide information) by using the machine learningmodel. According to yet another example embodiment, the processor 521may calculate a change rate of an expectation value for each of thecandidate operation commands by using the machine learning model. Theprocessor 521 may determine the operation command by calculating anoperation value having the largest change rate of the expectation valueamong the candidate operation commands. However, the determining of theoperation command using the machine learning model by the processor 521is not limited to those described above, and all machine learning modelscapable of determining the operation command from the blood vessel imagemay be used.

FIG. 6 is a diagram illustrating extracting partial guide informationaccording to an example embodiment.

An apparatus 600 for training a machine learning model 612 according toan example embodiment may include a processor 610 and a memory 620.After the medical tool control device performs a series of operationcommands, the processor 610 may receive, from the memory 620, partialguide information 621 of a first time frame, partial guide information622 of a second time frame, which is a next time of the first time, andoperation command data 623 that has been transmitted to the medical toolcontrol device at the first time. For example, if the first time frameis a t-th frame, the second time frame may be a t+1-th frame, in which tmay be an integer of 1 or more. The time unit of each time frame may bedetermined according to a design.

An evaluation data calculation unit 611 of the processor 610 maycalculate evaluation data based on the partial guide information 622 ofthe second time frame. The machine learning model 612 may receive thepartial guide information 621 of the first time frame, the partial guideinformation 622 of the second time frame, the operation command data 623at the first time, and the evaluation data as training data to updateparameters.

According to an example embodiment, the training data may be acombination of the partial guide information 621 and 622 of the firsttime frame and the second time frame generated when a series ofoperation commands and each operation command are output after a seriesof operations are performed. However, the training data is not limitedthereto, and may include combinations of the partial guide information621 and 622 of the first time frame and the second time frame generatedwhenever respective operation commands are performed. That is, inresponse to the performing of each operation command by the medical toolcontrol device, the memory 620 may store the operation command data 623,the partial guide information 621 of a time frame before the operationcommand is performed, and the partial guide information 622 of a timeframe after the operation command is performed may be stored as onetraining set.

The processor 610 according to an example embodiment may compensate theevaluation data by applying a compensation value calculated according toa comparison result between the position to which the distal end of themedical tool has been moved and the guide data to the evaluation data.The applying of the compensation value according to the guide data andthe position of the distal end of the medical tool will be described indetail with reference to FIG. 7 .

The processor 610 may update the machine learning model 612 based on theevaluation data to which the compensation value is applied. Theprocessor 610 may calculate an estimated evaluation value associatedwith the operation command from a guide image corresponding to the firsttime frame before the operation command is performed. In addition, theprocessor 610 may calculate a measured evaluation value from a guideimage corresponding to a time frame (e.g., a second time frame) afterthe distal end of the medical tool has been moved according to theoperation command output in the first time frame. The processor 610 maydetermine a parameter to update the machine learning model 612 by usingthe estimated evaluation value and the measured evaluation value.According to an example embodiment, the processor 610 may calculate, asthe estimated evaluation value, an expectation value calculated byperforming the operation command determined by the machine learningmodel 612 by the medial tool control device in the first time frame. Inaddition, the processor 610 may calculate a candidate expectation valueof each of the candidate operation commands which may be performed bythe medical tool control device in the second time frame, and calculatea value obtained by adding evaluation data to the largest candidateexpectation value among the candidate operation commands as a measuredevaluation value. Here, the expectation value may mean a cumulativecompensation expectation value that may be obtained when the medicaltool control device performs a series of operation commands.Accordingly, the estimated evaluation value may be a value indicating acumulative compensation expectation value before the medical toolcontrol device actually performs the operation command. The measuredevaluation value may be a value obtained by applying a compensationvalue obtained by performing the actual operation command to a maximumexpectation value in a time frame after the medical tool control deviceperforms the actual operation command.

The processor 610 may calculate a parameter to update the machinelearning model 612 based on a loss calculated by using the measuredevaluation value and the estimated evaluation value. For example, theprocessor 610 may update the parameters of the machine learning model612 such that a difference between the measured evaluation value and theestimated evaluation value as the loss is minimized. The processor 610may repeat the parameter updating of the machine learning model 612until the calculated loss is less than a threshold loss. In other words,the processor 610 may learn the machine learning model 612 so that theestimated evaluation value (e.g., a cumulative compensation expectationvalue estimated between the first time frame and the second time frame)is equal to or similar to the measured evaluation value (e.g., a valueobtained by applying the compensation value to the maximum expectationvalue calculated after the actual operation command is performed).

FIG. 7 is a diagram illustrating a guide image corresponding to acurrent time frame and a guide image corresponding to the next timeframe according to an example embodiment.

The processor according to an example embodiment may apply, toevaluation data, a compensation value calculated according to acomparison result between guide data and a position to which the distalend of the medical tool has been moved in the blood vessel image. Theprocessor may calculate a compensation value by comparing the bloodvessel image of the second time frame after the distal end of themedical tool has been moved from the blood vessel image of the firsttime frame.

The guide data generated from the blood vessel image may includeinformation about a position of the distal end of the medical tooltogether with at least one of a middle target point, an accessrestriction point, and a destination point in a patch imagecorresponding to at least a partial area of the blood vessel image asguide information. The processor according to an example embodiment maycalculate a compensation value according to a position to which thedistal end of the medical tool has been moved in the blood vessel patchimage in which the guide information is visualized.

The processor according to an example embodiment may apply a first groupcompensation value to the evaluation data in response to a case wherethe distal end of the medical tool reaches one of the destination pointand the middle target point in the blood vessel image. The first groupcompensation value may vary depending on a type of guide data. Forexample, the processor may assign a larger first group compensationvalue to a case where the distal end of the medical tool reaches thedestination point than a case where the distal end of the medical toolreaches the middle target point. The processor may apply the firstcompensation value to the evaluation data in response to the case ofreaching the destination point. In response to the case of reaching themiddle target point, the processor may apply a second compensation valuesmaller than the first compensation value to the evaluation data.

In addition, the processor according to an example embodiment may applya second group compensation value to the evaluation data when the distalend of the medical tool reaches the access restriction point in theblood vessel image or moves into a section between the areas. Althoughthe distal end of the medical tool did not reach any one of the middletarget point, the destination point, and the access restriction point,the case of moving into the section between the areas may be a case inwhich the medical tool control device performs an operation command. Thesecond group compensation value may vary depending on a type of guidedata. For example, the processor may assign a larger second groupcompensation value to a case where the distal end of the medical toolreaches the access restriction point than a case where the distal end ofthe medical tool moves into the section between the areas. The processormay apply a third compensation value to the evaluation data in responseto the case of reaching the access restriction point. The processor mayapply a fourth compensation value having a smaller absolute value thanthe first compensation value in response to the case of moving into thesection between the areas.

According to the example embodiment of FIG. 7 , the processor mayrecognize that the distal end of the medical tool has reached a firstmiddle target point 711 in a patch image 710 corresponding to a partialarea of the blood vessel image of the first time frame. Accordingly, theprocessor may determine an operation command of the medical tool controldevice based on the machine learning model. When the medical toolcontrol device moves the medical tool according to the operation commandof the processor, the processor may obtain a first patch image 720 or asecond patch image 730 in the second time frame according to a type ofoperation command.

For example, the first patch image 720 may represent an image obtainedwhen the distal end of the medical tool reaches the access restrictionpoint according to an arbitrary operation command. The processor maydetermine from the first patch image 720 that the distal end of themedical tool has reached an access restriction point 721. In response tothe case where the distal end of the medical tool reaches the accessrestriction point 721, the processor may apply a second groupcompensation value corresponding to the access restriction point to theevaluation data.

As another example, the second patch image 730 may represent an imageobtained when the distal end of the medical tool reaches a second middletarget point 731 according to another operation command. The processormay determine from the second patch image 730 that the distal end of themedical tool has reached the second middle target point 731. In responseto the case where the distal end of the medical tool reaches the secondmiddle target point 731, the processor may apply a first groupcompensation value corresponding to the middle target point to theevaluation data.

For reference, for convenience of description, it has been describedthat in the first patch image 720 of the second time frame from thepatch image 710 of the first time frame, the distal end of the medicaltool reaches the access restriction point 721 by one operation command,but it is not limited thereto. According to a distance between the firstmiddle target point 711 and the access restriction point 721, when thedistal end of the medical tool reaches from the first middle targetpoint 711 to the access restriction point 721, a plurality of operationcommands may be required. Similarly, it has been described that in thesecond patch image 730 of the second time frame from the patch image 710of the first time frame, the distal end of the medical tool reaches thesecond middle target point 731 by one operation command, but it is notlimited thereto. According to a distance between the first middle targetpoint 711 and the second middle target point 731, when the distal end ofthe medical tool reaches from the first middle target point 711 to thesecond middle target point 731, a plurality of operation commands mayalso be required. When the distal end of the medical tool does not reachthe middle target point by an operation according to one operationcommand, but moves only into the section between the areas, a negativecompensation value may be applied to the evaluation data as describedabove.

FIG. 8 is a block diagram illustrating a system for training a machinelearning model for determining an operation command of a medical toolcontrol device according to an example embodiment.

A system 800 for training a machine learning model according to anexample embodiment may include a medical tool control device 820, atraining device 810, and a blood vessel imaging device 830. The medicaltool control device 820 may include a processor 821, a memory 822, aninput/output interface, a driving unit 824, and a medical tool 825, andthe training device 810 may include a processor 811, a memory 812, andan input/output interface. The medical tool control device 820, thetraining device 810, and the blood vessel imaging device each mayperform each function with a separate housing, but are not limitedthereto, and may perform each function with a single housing.

The processor 811 of the training device 810 according to an exampleembodiment may train the machine learning model by using the evaluationdata calculated according to the position of the medical tool 825 in theblood vessel image. Since the training of the machine learning model hasbeen described above, a detailed description thereof will be omitted.The memory 812 of the training device 810 may at least temporarily storeat least one of guide data, operation commands, evaluation data, and amachine learning model, and the processor 811 may train the machinelearning model by using the data stored in the memory 812. Theinput/output interface of the training device 810 may be connected tothe medical tool control device 820 and the blood vessel imaging device830 to transmit and receive data.

The processor 821 of the medical tool control device 820 according to anexample embodiment may determine an operation of the driving unit 824based on the operation command received from the training device 810.The memory 822 of the medical tool control device 820 may at leasttemporarily store the data received from the training device 810 or theblood vessel imaging device 830, and may transmit the data to theprocessor 821. The input/output interface of the medical tool controldevice 820 may be connected to the training device 810 and the bloodvessel imaging device 830 to transmit and receive data. The driving unit824 of the medical tool control device 820 may guide a medical tool 825using a motor according to an operation command determined by theprocessor 821 of the medical tool control device 820.

The example embodiments described above may be implemented in hardwarecomponents, software components, and/or combinations of hardwarecomponents and software components. For example, the apparatus, themethod, and the components described in the example embodiments may beimplemented using, for example, one or more general-purpose computers orspecial-purpose computers, such as a processor, a controller, anarithmetic logic unit (ALU), a digital signal processor, amicrocomputer, a field programmable gate array (FPGA), a programmablelogic unit (PLU), a microprocessor, or other any devices capable ofexecuting and responding instructions. The processing device may performan operating system OS and one or more software applications performedon the operating system. In addition, the processing device may alsoaccess, store, manipulate, process, and generate data in response toexecution of software. For convenience of understanding, one processingdevice may be described to be used, but it can be seen to those skilledin the art that the processing device may include a plurality ofprocessing elements and/or a plurality of types of processing elements.For example, the processing device may include a plurality of processorsor one processor and one controller. In addition, other processingconfigurations, such as a parallel processor are also possible.

Software may include computer programs, codes, instructions, or one ormore combinations thereof, and may configure the processing device tooperate as desired, or to instruct independently or collectively theprocessing device. Software and/or data may be permanently ortemporarily embodied in any type of machines, components, physicaldevices, virtual equipment, computer storage media or devices, or signalwaves to be transmitted, in order to be interpreted by the processingdevice or provide instructions or data to the processing device. Thesoftware may be distributed on a computer system connected via anetwork, and may be stored or executed in a distributed method. Thesoftware and data may be stored in one or more computer readablerecording media.

The method according to the example embodiment may be implemented in aform of program instructions which may be performed through variouscomputer means to be recorded in computer readable media. The computerreadable media may include program instructions, data files, datastructures, and the like alone or in combination. The programinstructions recorded in the medium may be specially designed andconfigured for the example embodiments or may be publicly known to andused by those skilled in the computer software art. Examples of thecomputer readable media include magnetic media, such as a hard disk, afloppy disk, and a magnetic tape, optical media such as a CD-ROM and aDVD, magneto-optical media such as a floptical disk, and hardwaredevices such as a ROM, a RAM, and a flash memory, which are speciallyconfigured to store and execute the program instructions. Examples ofthe program instructions include high language codes executable by acomputer using an interpreter and the like, as well as machine languagecodes created by a compiler. The hardware devices may be configured tooperate as one or more software modules in order to perform theoperations of the example embodiments, and vice versa.

As described above, although the example embodiments have been describedby the restricted drawings, various modifications and variations can beapplied based on the example embodiments by those skilled in the art.For example, even if the described techniques are performed in adifferent order from the described method, and/or components such as asystem, a structure, a device, a circuit, and the like described aboveare coupled or combined in a different form from the described method,or replaced or substituted by other components or equivalents, anappropriate result can be achieved.

The invention claimed is:
 1. A method for training, by a processor, amachine learning model for determining an operation of a medical toolcontrol device comprising steps of: Obtaining, by a processor, anoperation command to move a medical tool of the medical tool controldevice based on the machine learning model from guide data generatedusing a blood vessel image; generating, by the processor, evaluationdata of a position to which a distal end of the medical tool has beenmoved according to the operation command in the blood vessel image; andupdating, by the processor, the machine learning model by changing aparameter of the machine learning model by using the evaluation data;wherein the processor generating of the evaluation data includes:applying a first group compensation value to the evaluation data inresponse to a case where the distal end of the medical tool reaches oneof the destination area and the middle target area in the blood vesselimage; and applying a second group compensation value to the evaluationdata in response to a case where the distal end of the medical toolreaches a penalty area and a case where the distal end of the medicaltool has been moved into a section between a middle target area, adestination area, or the penalty area.
 2. The method for training themachine learning model of claim 1, wherein the processor generating ofthe evaluation data includes applying a compensation value calculated byevaluating the position to which the distal end of the medical tool hasbeen moved in the blood vessel image based on the guide data to theevaluation data.
 3. The method for training the machine learning modelof claim 1, wherein the guide data includes information about theposition of the distal end of the medical tool together with at leastone of a middle target area, an access restriction point (a penaltyarea), and a destination area in a patch image corresponding to at leasta partial area of the blood vessel image.
 4. The method for training themachine learning model of claim 1, wherein the processor applying of thefirst group compensation value includes applying a first compensationvalue to the evaluation data in response to the case of reaching thedestination area; and applying a second compensation value distinguishedfrom the first compensation value to the evaluation data in response tothe case of reaching the middle target area.
 5. A non-transitorycomputer-readable recording medium storing one or more computer programsincluding instructions for performing the method of claim
 1. 6. Themethod for training the machine learning model of claim 1, wherein theprocessor applying of the second group compensation value includes:applying a third compensation value to the evaluation data in responseto the case of reaching the penalty area; and applying a fourthcompensation value distinguished from the third compensation value inresponse to the case of moving into the section between the areas. 7.The method for training the machine learning model of claim 1, whereinthe processor training of the machine learning model includes:calculating an estimated evaluation value associated with an operationcommand output from a guide image corresponding to a current time frameby using the machine learning model; calculating a measured evaluationvalue from a guide image corresponding to a next time frame after thedistal end of the medical tool has been moved according to the operationcommand output in the current time frame; and determining a parameter toupdate the machine learning model by using the estimated evaluationvalue and the measured evaluation value.
 8. The method for training themachine learning model of claim 1, wherein the processor obtaining ofthe operation command of the medical tool control device includes:calculating an expectation value to reach a destination area for each ofcandidate operation commands that may be performed by the medical toolcontrol device by using the machine learning model; and outputting anoperation command having the highest expectation value among thecandidate operation commands by using the machine learning model.
 9. Themethod for training the machine learning model of claim 1, comprising:obtaining, by the processor, a blood vessel structure from the bloodvessel image; setting a middle target area and a penalty area based onthe blood vessel structure and the destination area; obtaining an entireguide image in which the destination area, the middle target area, andthe penalty area are set; extracting partial guide information from theentire guide image based on the position of the distal end of themedical tool; and determining the operation command based on the machinelearning model from the partial guide information.
 10. An apparatus fortraining a machine learning model comprising: a processor configured toobtain an operation command for moving a medical tool of a medical toolcontrol device based on the machine learning model for determining anoperation of the medical tool control device from guide data generatedusing a blood vessel image, generate evaluation data for a position towhich a distal end of the medical tool has been moved according to theoperation command in the blood vessel image, and update the machinelearning model by changing a parameter of the machine learning model byusing the evaluation data; and a memory configured to store at least oneof the guide data, the operation command, the evaluation data, and themachine learning model, wherein the processor applies a first groupcompensation value to the evaluation data in response to a case wherethe distal end of the medical tool reaches one of the destination areaand the middle target area in the blood vessel image, and applies asecond group compensation value in response to a case where the distalend of the medical tool reaches a penalty area and a case where thedistal end of the medical tool has been moved into a section between amiddle target area, a destination area, or the penalty area.
 11. Theapparatus for training the machine learning model of claim 10, whereinthe processor obtains a blood vessel structure from the blood vesselimage, sets a middle target area and a penalty area based on the bloodvessel structure and a destination area, obtains an entire guide imagein which the destination area, the middle target area, and the penaltyarea are set, extracts partial guide information from the entire guideimage based on the position of the distal end of the medical tool, anddetermines the operation command based on the machine learning modelfrom the partial guide information.
 12. The apparatus for training themachine learning model of claim 10, wherein the processor applies acompensation value calculated by evaluating the position to which thedistal end of the medical tool has been moved in the blood vessel imagebased on the guide data to the evaluation data.
 13. The apparatus fortraining the machine learning model of claim 10, wherein the guide dataincludes information about the position of the distal end of the medicaltool together with at least one of a middle target area, a penalty area,and a destination area in a patch image corresponding to at least apartial area of the blood vessel image.
 14. The apparatus for trainingthe machine learning model of claim 10, wherein the processor applies afirst compensation value to the evaluation data in response to the caseof reaching the destination area and applies a second compensation valuedistinguished from the first compensation value to the evaluation datain response to the case of reaching the middle target area.
 15. Theapparatus for training the machine learning model of claim 10, whereinthe processor calculates an expectation value to reach a destinationarea for each of candidate operation commands that may be performed bythe medical tool control device by using the machine learning model andoutputs an operation command having the highest expectation value amongthe candidate operation commands by using the machine learning model.16. The apparatus for training the machine learning model of claim 10,wherein the processor applies a third compensation value to theevaluation data in response to the case of reaching the penalty area andapplies a fourth compensation value distinguished from the thirdcompensation value in response to the case of moving into the sectionbetween the areas.
 17. The apparatus for training the machine learningmodel of claim 10, wherein the processor calculates an estimatedevaluation value associated with an operation command output from aguide image corresponding to a current time frame by using the machinelearning model, calculates a measured evaluation value from a guideimage corresponding to a next time frame of the moving result accordingto the operation command output in the current time frame, anddetermines the evaluation data by using the estimated evaluation valueand the measured evaluation value.